Deployment of ML Model using Flask
by CM
Posted on May 15, 2020
The Goal:
In this article, we will use a (1) pre-trained machine learning model, (2) build a simple web-interface, and (3) deploy it using Flask web service. Let's clarify, this in more detail: We will use the machine learning model that we have created in the Disney Image Classification article. (Note: No need to train the model yourself -- it is provided under 'Key components' below). We will set up a simple HTML page as an interface to provide new images to the machine learning model and display the prediction output. We will use Flask to serve the machine learning model, as Flask is a micro web framework. We will use Flask as it is one of the most popular Python web application frameworks that is fairly easy to set up. For this article -- we will deploy our machine learning model locally. In the next article, we are going to leverage Google App Engine to host our Flask application.
Image classification:
Image classification is a supervised learning problem. We have defined a set of target classes (in our case Disney Princesses). Our model is capable of predicting which Disney Princess a given image generally looks like. The model is saved as a Keras model in hdf5 file format.
Key components are:
First things first, we need to install Flask on our machine. I highly recommend setting up a virtual environment (venv) for Flask respectively our Python application. The reason for having a dedicated venv is simple -- in order to deploy our application, we need to have respective requirements. Those requirements are simply our libraries and dependencies that are required to run the application, e.g., a specific TensorFlow version. In order to avoid dependency conflicts and to allow easy evaluation of the necessary app requirements for a future 'online' deployment e.g., via Google App Engine, having a virtual environment is highly recommended. The idea is here to pip freeze your requirements after your application works and use the saved requirements file on the cloud computing platform.
I particularly use virtualenv to set up my virtual environment. However, there are several options out there that can do a similar job. Let's go step by step now. We first going to install virtualenv. Then, we going to create the virtual environment in a selected folder. After that we are going to activate the new venv on our local host.
#First, install virtualenv:
#use conda instead of pip, in case you are using anaconda on your local machine.
#Explanation: You can now use virtualenv in your console.
pip install virtualenv
#Second, create the virtual environment -- do this in your console not in Python!
#Explanation: We just created a new virtual environment (my_project) in the current path.
virtualenv my_project
#Third, let's cd into the Scripts folder of our virtualenv and then activate it.
#Explanation: Your virtualenv should now be running.
cd script
activate
When everything worked out, then you should have four folders in your file explorer as shown below:
Looking at the console --- your virtual environment should be activated.
Next thing is that we need to install Flask. Therefore, just use pip to install the web framework.
#Do this in your console (not in python)
pip install flask
Ok for now we are set --- let's get into the code. First, we need to import our dependencies. If you are using a virtual environment, make sure to install the required dependencies upfront using pip in your console. In case you are missing some dependencies - check the console log output for the missing modules. For our application, we will need various dependencies: (1) For decoding / encoding our image, (2) for importing our pre-trained ml model, (3) for loading and running the pre-trained ml model, (4) for running flask and connecting the webpage respectively sending data.
import base64
import numpy as np
import tensorflow as tf
import io
from io import BytesIO
from PIL import Image
import keras
from tensorflow.keras import backend as K
from tensorflow.keras.models import Sequential
from tensorflow.keras.models import load_model
from flask import request
from flask import jsonify
from flask import Flask
Next, will make the app available to our Flask server. In other words, we will need to create the Flask instance. __name__ is the name of the current Python module. The app needs to know where it’s located to set up some paths, and __name__ is a convenient way to tell it that. If you are using a single module (as in this example), you should use __name__ because depending on if it’s started as application or imported as module the name will be different ('__main__' versus the actual import name). This is needed so that Flask knows where to look for templates, static files, and so on. In other words, we are instantiating a Flask object by passing __name__ argument to the Flask constructor. The Flask constructor has one required argument which is the name of the application package. Most of the time __name__ is the correct value. The name of the application package is used by Flask to find static assets, templates and so on. For more information have a look at the Flask documentation.
app = Flask(__name__)
Next, we are going to import our model (which was pre-trained and saved as an hdf5 file. In this regard, make sure to know the file path of your model as we need to specify it when using the tf.keras.models.load_model() function. We will use a global variable to store the model. In Python, a variable declared outside of the function or in global scope is known as a global variable. This means that a global variable can be accessed inside or outside of the function.
class="python hljs"def get_model():
#Define Model variable
global model
#Use the file path of your model (in case it lies in root - you can just input the model name)
model = load_model('model.h5')
print(" * Keras-Model loaded successfully!")
return model
#We run our get_model function
get_model()
Run the code to make sure the model can be loaded without any problems. In case you have problems loading the model using load_model() - check your TensorFlow version. The model, in particular, was created / saved using TensorFlow 2.0.0b0. In this regard, you might need to downgrade / upgrade your TensorFlow version respectively. You can do this via pip install --upgrade tensorflow==2.0.0.b0.
==========================
OUTPUT
==========================
* Keras-Model loaded successfully!
We then use the route() decorator to tell Flask what URL should trigger our function. The function is given a name which is also used to generate URLs for that particular function, and returns the message we want to display in the user’s browser. Just save it as ml_app.py or something similar. Make sure to not call your application flask.py because this would conflict with Flask itself.
@app.route("/predict", methods = ["POST"])
Here we go, writing the code for our prediction. Upfront, what we plan on doing is to upload a picture to our web interface. We then plan to request this picture by our ml_app application. Therefore, we need to encode our image before we can send it to our application. We will do this later via JavaScript directly in the browser. The requested encoded image will then need to be decoded by our ml_app. We will transfer the image as a json for convenience. After receiving and encoding the image file, we need to convert it to a tensor that fits our machine learning model for image classification. This includes the conversion process to a tensor, normalizing the data, resizing the tensor, as well as expanding the dimensionality. Lastly, we will return our predictions as a json to the web interface. Luckily this all can be done in Python.
def predict():
message = request.get_json(force=True)
print("Message: ", message)
encoded = message['image']
#Add padding and decode base64
encoded += "=="
decoded = base64.b64decode(encoded)
print("Decoded: ", decoded)
#Convert decoded into Tensor
image = tf.image.decode_image(decoded, channels=3)
print("TF Image Decoded: ", image)
print("Image Shape: ", image.shape )
#Convert to Float
image = tf.image.convert_image_dtype(image, tf.float32)
print("TF Image Converted to Float: ", image)
print("Image Shape: ", image.shape )
#Resize Tensor
image = tf.image.resize(image, [150, 150])
print("TF Image Resize: ", image)
print("Image Shape: ", image.shape )
#Expanding Tensor (1, 150, 150, 3)
processed_image = np.expand_dims(image, axis=0)
print("TF Image Resize: ", processed_image)
print("Image Shape: ", processed_image.shape )
prediction = model.predict(processed_image).tolist()
response = {
'prediction': {
'anna':prediction[0][0],
'ariel':prediction[0][1],
'aurora':prediction[0][2],
'belle':prediction[0][3],
'cinderella':prediction[0][4],
'elsa':prediction[0][5],
'jasmine':prediction[0][6],
'merida':prediction[0][7],
'moana':prediction[0][8],
'mulan':prediction[0][9],
'pocahontas':prediction[0][10],
'rapunzel':prediction[0][11],
'snow':prediction[0][12],
'tiana':prediction[0][13]
}
}
print(response)
return jsonify(response)
Now it is time to build the web interface. In this regard, for the purpose of demonstration, we will simply set up a plain html page with an input and prediction button that serve for uploading a picture and sending the encoded image to our ml_app, as well as displaying the results. First, we will create the header of our HTML file and importing jQuery (which is needed to handle the post method) hosted on Google CDN.
<!DOCTYPE html>
<html lang="en">
<head>
<title> Disney Image Classification with Flask </title>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
</head>
Next up, we will include some very basic style for our button and adjust font-size for better readability.
<style>
.button {
display: inline-block;
padding: 15px 25px;
font-size: 24px;
cursor: pointer;
text-align: center;
text-decoration: none;
outline: none;
color: #fff;
background-color: #4CAF50;
border: none;
border-radius: 15px;
box-shadow: 0 9px #999;
}
.button:hover {background-color: #3e8e41}
.button:active {
background-color: #3e8e41;
box-shadow: 0 5px #666;
transform: translateY(4px);
}
#image-selector {
border: 1px solid #ccc;
display: inline-block;
padding: 6px 12px;
cursor: pointer;
}
input[type="file"] {
display: none;
}
p {font-size: 30px}
</style>
Time to build the body of our web interface. First, we will create the picture upload button.
<body>
<h1 style="font-size:36px; text-align:center"> Disney Princesses Prediction ML Model!</h1>
<div style="width:1000px; height:100px; text-align: center;">
<label for="image-selector" class="custom-file-upload">
<i class="fa fa-cloud-upload"></i>
</label>
<input style=" width: 20em; height: 2em;" id ="image-selector" type="file"/>
</div>
Next up, we are going to display our image upload and build our predict button.
<div style="padding:10px; text-align: center;">
<img id="selected-image" src="" width="200" height="200"/>
</div>
<button class="button" style="width: 100%; height:120px" id="predict-button">Predict Image
Now we need a placeholder to display our prediction results for each classification category. Remember, we are predicting Disney Princesses in our ML example.
<p>Anna <span id = "anna-prediction"> </span> </p>
<p>Ariel <span id = "ariel-prediction"> </span> </p>
<p>Aurora <span id = "aurora-prediction"> </span> </p>
<p>Belle <span id = "belle-prediction"> </span> </p>
<p>Cinderella <span id = "cinderella-prediction"> </span> </p>
<p>Elsa <span id = "elsa-prediction"> </span> </p>
<p>Jasmine <span id = "jasmine-prediction"> </span> </p>
<p>Merida <span id = "merida-prediction"> </span> </p>
<p>Moana <span id = "moana-prediction"> </span> </p>
<p>Mulan <span id = "mulan-prediction"> </span> </p>
<p>Pocahontas <span id = "pocahontas-prediction"> </span> </p>
<p>Rapunzel <span id = "rapunzel-prediction"> </span> </p>
<p>Snow <span id = "snow-prediction"> </span> </p>
<p>Tiana <span id = "tiana-prediction"> </span> </p>
What is left now is to create the JavaScript functionality regarding the image encoding, the request functionality, and well as formatting and displaying the prediction results.
<script>
let base64Image;
$("#image-selector").change(function() {
let reader = new FileReader();
reader.onload = function(e) {
let dataURL = reader.result;
$('#selected-image').attr("src", dataURL);
base64Image= dataURL.replace(/^data:image\/(png|jpeg);base64,/, "");
//alert(base64Image);
}
reader.readAsDataURL($("#image-selector")[0].files[0]);
$("#anna-prediction").text("");
$("#ariel-prediction").text("");
$("#aurora-prediction").text("");
$("#belle-prediction").text("");
$("#cinderella-prediction").text("");
$("#elsa-prediction").text("");
$("#jasmine-prediction").text("");
$("#merida-prediction").text("");
$("#moana-prediction").text("");
$("#mulan-prediction").text("");
$("#pocahontas-prediction").text("");
$("#rapunzel-prediction").text("");
$("#snow-prediction").text("");
$("#tiana-prediction").text("");
});
$("#predict-button").click(function(event){
alert("Encoding of Image --> Decode --> ML prediction --> Sending information back to HTML");
let message = {
image: base64Image
}
console.log(message);
$.post("http://localhost:5000/predict", JSON.stringify(message), function(response){
$("#anna-prediction").text(parseFloat(response.prediction.anna).toFixed(2)*100+"%");
$("#ariel-prediction").text(parseFloat(response.prediction.ariel).toFixed(2)*100+"%");
$("#aurora-prediction").text(parseFloat(response.prediction.aurora).toFixed(2)*100+"%");
$("#belle-prediction").text(parseFloat(response.prediction.belle).toFixed(2)*100+"%");
$("#cinderella-prediction").text(parseFloat(response.prediction.cinderella).toFixed(2)*100+"%");
$("#elsa-prediction").text(parseFloat(response.prediction.elsa).toFixed(2)*100+"%");
$("#jasmine-prediction").text(parseFloat(response.prediction.jasmine).toFixed(2)*100+"%");
$("#merida-prediction").text(parseFloat(response.prediction.merida).toFixed(2)*100+"%");
$("#moana-prediction").text(parseFloat(response.prediction.moana).toFixed(2)*100+"%");
$("#mulan-prediction").text(parseFloat(response.prediction.mulan).toFixed(2)*100+"%");
$("#pocahontas-prediction").text(parseFloat(response.prediction.pocahontas).toFixed(2)*100+"%");
$("#rapunzel-prediction").text(parseFloat(response.prediction.rapunzel).toFixed(2)*100+"%");
$("#snow-prediction").text(parseFloat(response.prediction.snow).toFixed(2)*100+"%");
$("#tiana-prediction").text(parseFloat(response.prediction.tiana).toFixed(2)*100+"%");
});
});
</script>
</body>
</html>
To run the application you can either use the Flask command or Python’s -m switch with Flask. Before you can do that you need to tell your console the application to work with by setting (Windows) / exporting (Linux) the FLASK_APP environment variable:
$ export FLASK_APP=ml_app.py
$ flask run --host=0.0.0.0
Let's run our application by going to the web interface. For demonstration purposes we are going to upload a picture of Princess Elsa.
Ok -- clicking the predict button. And wait for the magic. Remember, the image gets now encoded by our JavaScript function than pushed to the ml_app python application, then gets decoded, then converted to a tensor that our model can use as an input, the outcoming predictions will then be pushed back to the web interface and displayed to us again. The entire process should take a couple of second including the prediction, requests, and decoding respectively encoding.
.
This worked perfectly. Here we go -- Princess Elsa!! We can identify you! No it is time to upload your pictures! How do You Look Like?